Appl. No. 09/916,249 

Amdt dated June 4, 2004 

Reply to Office Action of Feb. 4, 2004 



REMARKS / ARGUMENTS 

Claims 1-18 are pending in this application. Claims 1-18 stand rejected. 

Upon entry of this amendment, independent claims 1-18 have been 
canceled, with new claims 19-23 added. 

In view of the following discussion, the applicant submits that all of the 
claims, as they now stand, are allowable. 

If, however, the Examiner believes that there are any unresolved issues 
requiring adverse action in any of the claims now pending in the application, it is 
requested that the Examiner telephone Jeffery J. Brosemer, Ph.D., ESQ. at 732- 
275-9167 so that arrangements may be made for resolving such issues as 
expeditiously as possible. 



Objections: 

Objection to the Drawing 

As noted prior, the Drawing was objected to for a number of errors. In 
response, the applicants have submitted with this Amendment, a proposed set of 
drawing changes wherein the proposed changes are made in red ink. 

Objection to the Claims 

Claims 1, 7, and 13 were objected to for a number of informalities. With 
this amendment, these claims have been canceled and, as such, the Applicant 
respectfully requests the Examiner to withdraw these objections. 
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Rejections: 

Claim Rejections - 35 U.S.C. § 101 

Claim 1 was rejected by the Examiner as being directed to non-statutory 
subject matter. In particular, the Examiner states that claim 1 offered abstract 
ideas not applied to the technological arts. In addition, the Examiner stated that 
claim 1 was without any particular practical application, such as a program 
running on a computer and stored in a computer readable medium or memory 
and that the claimed invention is not supported by a credible asserted utility or 
well established utility. 

As noted before, claim 1 has been canceled by this amendment. Newly 
presented claim 19, is directed to a method implemented as a computer-based 
system for recognizing events in video sequences. As such, the invention now 
claimed is statutory. 

Claim Rejections - 35 U.S.C. § 112 

Claim 1 was rejected by the Examiner under the provisions of 35 U.S.C. § 
112, first paragraph as failing to comply with the enablement requirement. In 
addition, the Examiner rejected claims 12 and 18 under the provisions of 35 
U.S.C. § 112, second paragraph as being indefinite. 

With this amendment, claims 1, 12 and 18 have been canceled. The 
applicant submits that new claims now presented, fully comply with the 
provisions of 35 U.S.C. 1 1 2. 
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Claim Rejections - 35 U.S.C. § 102 

Claims 1, 7 and 13 were rejected under the provisions of 35 U.S.C. § 102 
as being anticipated by United States Patent No. 3,647,978 that issued to Hill on 
March 7, 1972 for Speech Recognition Apparatus (hereinafter referred to as "Hill 
'978 patent"). 

Inasmuch as claims 1,7, and 13 have been canceled by this Amendment, 
the rejection with respect to the canceled claims is now moot. However, in order 
to expedite the examination of the instant application with respect to this 
Amendment and the newly presented claims, discussions pertinent to these 
rejections will be presented along with those particular to the rejections under the 
provisions of 35 U.S.C. § 103, which follows. 



Claim Rejections - 35 U.S.C. § 103 

Claims 2-6, 8-12, and 14-18 were rejected under the provisions of 35 
U.S.C. § 103(a) as being unpatentable over the Hill '978 patent in view of United 
States Patent No. 6,021,403 issued to Horvitz on February 1, 2000 (hereinafter 
the Horvitz '403 patent) further in view of an article entitled "Grounding Language 
In Perception", authored by the inventor of the instant application that appeared 
in Artificial Intelligence Review, Volume 8, pp. 371-391, 1995 (hereinafter 
"Siskind"). 

With this Amendment, the applicants have canceled the rejected claims 
and presented in their place, a new set of claims. In view of these amendments 
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and the following discussion, the applicant submits that the new claims are 
allowable. 

Before we discuss the cited art in detail however, it is useful to review the 
claimed invention of the present application. 

Generally, the present invention is directed to a method for computing all 
occurrences of a compound event from occurrences of primitive events where 
the compound event is a defined combination of the primitive events. The 
method comprises the steps of: (a) defining primitive event types; (b) defining 
combinations of the primitive event types as a compound event type; (c) inputting 
the primitive event occurrences, such occurrences being specified as the set of 
temporal intervals over which a given primitive event type is true; and (d) 
computing the compound event occurrences, such occurrences being specified 
as the set of temporal intervals over which the compound event type is true, 
where the set of temporal intervals in steps (c) and (d) above, are specified as 
smaller sets of spanning intervals, each spanning interval representing a set of 
intervals. 

As described and now claimed, the methods of the present invention are 
implemented in a system for recovering event occurrences from video input. It 
differs from prior art approaches in two significant ways. First, it employs state 
changes in force-dynamic relations between objects, instead of motion profile as 
a key descriptive element in defining event types; and Second, it employs event 
logic, instead of hidden Markov models, to perform event classification. 
Additionally, and as only now taught by the present invention, is the use of 
spanning intervals that efficiently represent the infinite sets of intervals that result 
when processing events. Consequently, the present invention results in an 
efficient procedure, based upon spanning intervals, for inferring all occurrences 
of compound event types from the occurrences of primitive event types. 
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In operation, the invention of the present application takes relatively short 
(30 to 120 frame) video sequences as input. These video sequences typically 
depict a person performing various actions with colored blocks, the actions 
comprising a set compound events such as pick up, put down, stack, unstack, 
move, assemble, and disassemble. These compound events are specified as 
event-logic expressions over primitive event types depicted in the video 
sequences, wherein the primitive event types include supported, rigidly attached, 
supports, contacts, and attached. 

The intervals over the input video sequences during which specific 
primitive events hold, are computed and represented by spanning intervals as 
now taught by the present invention. The compound event types are then 
determined from the primitive event(s). 

Turning our attention now to the cited prior art, and in particular the Hill 
'978 patent, there the patentee discloses a speech recognition apparatus that is 
responsive to selected acoustic characteristics for decomposing a signal 
representing an acoustic input into analog signals on parallel channels. The 
analog signals may then be transformed into binary signals on parallel channels 
which constitute time-ordered event markers. Sequential properties of the binary 
signals may be marked and nonordered array pattern binary information 
representing both content and order information relating to the acoustic input 
may then be stored. 

And while the Examiner is certainly correct that certain, broad teachings of 
the Hill '987 patent are relevant to the claimed invention of the instant application, 
the Examiner is mistaken in his broad application of the teachings. 
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Specifically, the Hill '987 patent provides a system for determining and 
recording two varieties of time-dependent information found in speech, namely 
the duration of speech events and the order of events. It is explicitly taught by 
the Hill '987 patent that these two types of time information should be thought of 
differently and that the handling of duration information is best handled during 
acoustic analysis. As taught by the patentee, the output of acoustic analysis is a 
set of data lines that carry data as standard pulses, and each channel may be 
derived from a circuit responsive to some particular characteristic of speech, 
called a Primitive Acoustic Characteristic (PAC), determined from the acoustic 
analysis. The inclusion of threshold information identifies what is called Primitive 
Acoustic Features (PAF), occurring in the PAC's. Finally, the beginning and 
ending of a PAF are themselves events which are noted, since they are 
significant parts of a PAF, and such events are called Primitive Acoustic Events 
(PAE). 

It is these PAE's to which input signals are reduced. Resolution of order is 
determined by the width of the pulse representing such events. Further 
processing of the information may produce additional aspects of the sequence of 
events in terms of structural descriptors called compound acoustic events 
(CAE's). 

At this point it should be readily apparent that the PAE's are NOT the 
same as the primitive event types as taught and claimed in the instant 
application. More specifically, PAE's as taught by the Hill '978 patent identify the 
beginning and end of PAF's, thereby delineating them, while the primitive events 
in the instant application pertain to the relationships of objects contained within 
scenes depicted in a series of video sequences. 

Consequently, the compound acoustic events (CAE's), which are defined 
in terms of CAE's or PAE's only - and therefore a series of PAF delineations, 
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ARE NOT the compound events in the instant application. More particularly, the 
compound events of the instant application are specified as event-logic 
expressions over the primitive event types, and result in events such as pick up, 
put down, stack, unstack, move, assemble, and disassemble. 

Recall for a moment that the present invention uses spanning intervals - 
which were heretofore unknown in the art for problems such as this - for 
efficiently representing the infinite sets of intervals that arise when processing 
liquid and semi-liquid events. 

As can be appreciated, when using event logic, it is desirable to compute 
and represent the set of all intervals over which some event-logic expression 
holds. Many primitive event types, including all of the primitive event types 
described in the instant application, are liquid in the sense that if an event holds 
for an interval then that event holds for every subinterval of that interval as well. 
With real-valued interval endpoints, this creates the need to compute and 
represent an infinite set of intervals for a liquid event, such as those in the instant , 
application. Even when limited to integer-valued endpoints, a liquid event will 
require the computation and representation of quadratically many endpoints. 

The applicant of the instant application addresses this problem in a novel 
manner through the use of spanning intervals, which represents the set of all 
subintervals in a computationally practical manner. 

In rejecting the original claims however, the Examiner states that the 
Horvitz '403 patent teaches spanning intervals and when combined with the 
teachings of the Hill '978 patent, makes obvious the present invention. The 
Examiner is mistaken, as neither the Hill'978 patent nor the Siskind reference 
teach the use of spanning intervals as first taught by the applicant. 
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Specifically, and with reference to the Horvitz '403 patent, there is taught a 
system for providing assistance with user interfaces found in computing systems. 
In particular, the Horvitz '403 patent teaches an event composing and monitoring 
system that allows high-level events to be created from combinations of low-level 
events. And while it too does teach several aspects relevant to the present 
invention, it DOES NOT teach the use of spanning intervals, as first taught and 
employed by the applicants of the instant application. 

In particular, the Examiner cites column 13, lines 33-60 as providing a 
disclosure of spanning intervals. Yet a careful review of that reference ONLY 
reveals an exemplary record format for modeled events in a modeled event 
database. More particularly, it indicates a number of record fields and their data 
type, it does not , either explicitly or implicitly, teach or describe a spanning 
interval which represents a set of all subintervals . 

Still further, it does not teach or suggest the use of a spanning interval to 
represent the set of all intervals over which a primitive event type holds and to 
compute and represent the set of all intervals over which compound event types 
hold. 

As the Examiner can readily appreciate, since the prior art does not teach 
or suggest all of the limitations of the claimed invention, it cannot anticipate or 
make obvious that claimed invention. 

Accordingly, the applicant submits that independent claim 19 is 
patentable. Inasmuch as the dependent claims each recites further 
distinguishing aspects of the present invention, the applicant submits that each of 
the dependent claims is non-obvious as well. 



Page 16 of 17 



Appl. No. 09/916,249 

Amdt. dated June 4, 2004 

Reply to Office Action of Feb. 4, 2004 

Conclusions: 

The applicants submit that all of the claims in their present form are 
allowable. Accordingly, both reconsideration of this application and its swift 
passage to issue are earnestly solicited. 



Respectfully submitted, 
JEFFERY J. BRQSEMER 




Reg. No. 36,096 
Tel.:(732)275.9167 
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